Recruiting Decay for Dynamic Power Reduction in Set-Associative Caches

In this paper, we propose a novel approach to reduce dynamic power in set-associative caches that leverages on a leakage-saving proposal, namely Cache Decay. We thus open the possibility to unify dynamic and leakage management in the same framework. The main intuition is that in a decaying cache, dead lines in a set need not be searched. Thus, rather than trying to predict which cache way holds a specific line, we predict, for each way, whether the line could be live in it. We access all the ways that possibly contain the live line and we call this way selection. In contrast to way-prediction, way-selection cannot be wrong: the line is either in the selected ways or not in the cache. The important implication is that we have a fixed hit time — indispensable for both performance and ease-of-implementation reasons. One would expect way-selection to be inferior to sophisticated way-prediction in terms of the total ways accessed, but in fact it can even do better. To achieve this level of accuracy we use Decaying Bloom filters to track only the live lines in ways — dead lines are automatically purged. We offer efficient implementations of such autonomously Decaying Bloom filters, using novel quasi-static cells. Our prediction approach affords us high-accuracy in narrowing the choice of ways for hits as well as the ability to predict misses — a known weakness of way-prediction — thus outperforming sophisticated way-prediction. Furthermore, our approach scales significantly better than way-prediction to higher associativity. We show that decay is a necessary component in this approach — way-selection and Bloom filters alone cannot compete with sophisticated way-prediction. We compare our approach to Multi-MRU and we show that without even considering leakage savings — we surpass it terms of relative power savings and in relative energy-delay in 4-way (9%) and more so in 8-way (20%) and 16-way caches (31%).